To support Build Faster, Artefact Builds, and Web Extension-registered metrics in Project FOG, we will need some mechanism for permitting the runtime registration of metrics.

Build Faster/Artefact Builds should be allowed to add metrics that (privileged) Javascript can record to and test without requiring a native compilation step.

Web Extensions should be allowed to have their own metrics.yaml files that glean_parser can then generate JS APIs from. These metrics should be usable in Firefox even if the addon is loaded at runtime.

This bug is about coming up with a design for these two similar but distinct use cases, seeking out and getting feedback from subject matter experts, and closing out the feedback cycle with a design we can implement later.

Let the drafting begin: https://docs.google.com/document/d/1-QvwR83Ss6Syyk5qzd1S4o0lgnxG6KotqirZAi8fGes/edit

Alrighty, it's in a state for a preliminary check. Please leave your comments in the doc before Monday Nov 9: https://docs.google.com/document/d/1-QvwR83Ss6Syyk5qzd1S4o0lgnxG6KotqirZAi8fGes/edit#

Left comments on the doc. I think the build faster section might benefit from additional details about how this is going to work. I also believe the APIs should rely on Glean.js (by swapping the generated code) for privileged addons, too.

Left a bunch of questions, just in time.

Oops, forgot to clear my ni? (but got a chance to read additional EU comments :)

Okidoki, the initial round of feedback has been incorporated. Now time to open it up to a wider audience.

:nalexander, you expressed an interest in how FOG might handle Artefact builds. Please provide your feedback in the doc and add whoever you know who might be interested. I'd like to tie a bow in this design (no doubt further discussion will happen as the impl fills in the blanks) by December 5, if that's okay.

Jared, you agreed to take a look, and for that I'm grateful.

I'll extend the feedback deadline to December 9 to give you space to work.

Here's the doc: https://docs.google.com/document/d/1-QvwR83Ss6Syyk5qzd1S4o0lgnxG6KotqirZAi8fGes/edit#heading=h.j21821piqudc

You can find me in the usual places with any questions.

I've added some high-level questions to the doc.

(In reply to Chris H-C :chutten from comment #6)

Okidoki, the initial round of feedback has been incorporated. Now time to open it up to a wider audience.

:nalexander, you expressed an interest in how FOG might handle Artefact builds. Please provide your feedback in the doc and add whoever you know who might be interested. I'd like to tie a bow in this design (no doubt further discussion will happen as the impl fills in the blanks) by December 5, if that's okay.

I added some comments and questions to the document, and Chris and I discussed this problem over video chat.

High-level comments:

1. providing some story for high-value Web Extensions (search experiments, say) to interoperate with Firefox Telemetry sounds valuable, but we should confirm it actually is valuable. Can we use Web Extension Experiment APIs for this use case?

2. for artifact builds, is the juice worth the squeeze? I.e., do we have enough existing consumers of artifact builds + changes to {Events,Scalars}.yaml to bother addressing this use case?

3. if the answer to 2) is yes, then there's a reasonable path to doing what Firefox does right now, and it can be smoothly upgraded to be more intelligent in the future as the need demands, so I have no substantive technical concerns with the proposal.

I am happy to provide guidance and reviews around the actual artifact build process, etc. Good work, Chris!

From a quick reactji poll and resultant discussion (on a Friday in December, no less!) it seems as though at least a handful of Slack#firefox devs have used (or have tried to use) Artifact Builds to add Telemetry, even as recently as a month ago. And from mach's build metrics (on by default for mozilla FTEs, on by explicit opt-in from contributors) we apparently have 15% of builds (20% of successful builds) being done in Artifact mode these days.

I've also reached out via email to some specific Firefox devs and have gotten at least one enthusiastic request for Artifact Build support.

So there exists a population of not inconsiderable size that already use what support we have in Firefox Telemetry. Knowing its size and enthusiasm, I can now make a better judgment call on how much complexity we should build into FOG to support it.

Thank you all for your feedback. There are some interesting discussions that I look forward to revisiting in detail when it comes to implementation time, but it seems as though in broad strokes the design is sound.